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CENTRAL LIMIT THEOREMS FOR A 
HYPERGEOMETRIC RANDOMLY REINFORCED URN 

IRENE GRIMALDI 


Abstract. We consider a variant of the randomly reinforced urn where more 
balls can be simultaneously drawn out and balls of different colors can be 
simultaneously added. More precisely, at each time-step, the conditional di¬ 
stribution of the number of extracted balls of a certain color given the past is 
assumed to be hypergeometric. We prove some central limit theorems in the 
sense of stable convergence and of almost sure conditional convergence, which 
are stronger than convergence in distribution. The proven results provide 
asymptotic confidence intervals for the limit proportion, whose distribution is 
generally unknown. Moreover, we also consider the case of more urns subjected 
to some random common factors. 


1. Introduction 

Urn models, also known as preferential attachment models, are stochastic pro¬ 
cesses in which, along the time-steps, different individuals or objects or categories 
(represented by different colors) receive some quantity, called “weight” (represented 
by the number of balls), in such a way that the higher the total weight they already 
have until a certain time, the greater the probability of receiving an additional 
weight at the next time (i.e. a “self-reinforcing” property). The preferential at¬ 
tachment is a key feature governing the dynamics of many biological, economic and 
social systems. Therefore, urn models are a very popular topic because of their 
hints for theoretical research and their applications in various fields: clinical trials 
(e.g. [SI Hina El 1371ES]), economics and finance (e.g. [71 nil [30]), information 
science (e.g. [35l|36]), network theory (e.g. [a[Hilo]) and so on. 

The first example of urn scheme is the standard Eggenberger-Polya urn |261I41] : 
an urn contains a red and h black balls and, at each discrete time, a ball is drawn 
out from the urn and then it is put again inside the urn together with an additional 
constant number fc > 0 of other balls of the same color. Let Zn be the proportion 
of red balls at time n, namely, the conditional probability of drawing a red ball at 
time n -|- 1, given the outcomes of the previous extractions. A well known result 
(see, for instance, [35]) states that (Z„) is a bounded martingale and converges 
almost surely to a random variable Z with Beta distribution with parameters a/k 
and b/k. 

Subsequently, urn models have been widely studied by many researchers and 
there is a rather extensive literature on them (e.g. [2 Slia H HSl ttllllll [21 HiMl 
I3H1I11I): a large number of new “replacement policies” (for instance, balanced rules, 
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tenable mechanisms and random reinforcements) and various related models (for 
instance, the Poisson-Dirichlet model and the very recent Indian buffet model 
i) have been introduced and analyzed from different points of view and by means 
of different techniques (combinatorial methods, martingales, branching processes, 
stochastic approximations, etc.). We refer to [40], and the references therein, for a 
general survey on random processes with reinforcement. 

In particular, as an extension of the Polya urn, the Randomly Reinforced Urn 
(RRU) was recently proposed and analyzed jU [HI [101 (HI HU (HI [^ lOZl ISHl SOI IH] ■ 
It consists in a multicolor urn which is reinforced at each time with a random num¬ 
ber of additional balls according to the color of the extracted ball. The distribution 
of the reinforcements may depend on time and be different for the different colors. 
These models are suitable in order to describe the evolution of some system, such as 
a population, and also to perform an adaptive design, i.e. an experimental design 
that uses accumulated data to decide on how to carry on the study, without under¬ 
mining the validity and the integrity of the experiment. Indeed, the RRU model 
provides randomized treatment allocation schemes (clinical trials) where patients 
are assigned to the best treatment with probability converging to one [IQIET]. 

In [3] a new version of the RRU model is formulated. This model consists of an 
urn which contains balls of two different colors, say a G N \ {0} balls of color A and 
b G N\ {0} balls of color B. At each time n > I, we simultaneously (i.e. without re¬ 
placement) draw a random number Nn of balls. Let A„ be the number of extracted 
balls of color A. Then we return the extracted balls in the urn together with other 
RnXn balls of color A and i?„(iV„ — A„) balls of color B. The size of the rein¬ 
forcement is assumed independent of ... ,Nn-i,Xn-i,Rn-i,Nn,Xn]. 

We will call this model “Hypergeometric Randomly Reinforced Urn” (HRRU). 

With respect to the RRU model, the main novelties of this model are that, at 
each time, more balls can be simultaneously drawn out (and returned in the urn) 
and balls of different colors can be simultaneously added. The number of extracted 
balls of a certain color depends on the composition of the urn at the moment of 
the extraction, akin a preferential attachment rule. When iV„ = 1 for each n, the 
HRRU reduces to the RRU model with equal reinforcements for the two colors. 
In particular, the case W = 1 and i?„ = k (where A: is a constant) for each n 
corresponds to the standard Eggenberger-Polya urn; while the case Nn = h and 
Rn = k (where h and k are two constants) for each n coincides with the model 
in [laHS]. Also the model introduced and studied in [^ can be seen as a RRU 
model where balls of different colors can be simultaneously added, but there the 
“multi-updating” is due to a delay in the updating. Indeed, at each time n a single 
ball is drawn out (and returned in the urn) but the updating is performed at certain 
time-steps (ui)i>i as follows: at time Ui, we add a random number i?„ of balls of 
the same color of the ball extracted at time n, for each n = Ui-i -I- 1,..., r^, with 

Ti < Ui. 

As explained in [U , a possible interpretation of the HRRU model is the following. 
At each time n > 1, a new firm appears on the market and it has to choose the 
operative system for its computers among two different types, say operative system 
A (to which we associate color A) and operative system B (to which we associate 
color B). The total number of its computers is RnNn (more precisely, Nn blocks of 
Rn computers each). The firm decides to adopt A„ blocks (of size each) with 
operative system A and {Nn — Xn) blocks (of size i?„ each) with operative systems 
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B, according to the number of computers with operative systems A already present 
in the market. Another possible interpretation follows. At each time n > 1, a 
pharmaceutical firm has to select the size of its production for two different kinds 
of products, say product A and product B. For instance, A and B can be two 
medicines for the same disease but with different costs. The total of its production 
is RnNn (more precisely, the firm produces Nn blocks, each of size i?„). The firm 
decides to produce A„ blocks of type A-products and {N^ — A„) blocks of type 
B-products according to the number of type A-products already on the market. 
Finally, setting i?„ = 1 for each n, the HRRU model can be employed to describe 
the growth of a population in which we can distinguish two types of individuals, 
say A and B. At each time n, the random numbers Nn and Xn represent the new 
offsprings and the new offsprings of type A, respectively. The number of the new 
type A-individuals depends on the composition of the population at the preceeding 
time-step. 

It is shown in [3], that Zn converges almost surely to a random variable Z, whose 
distribution is generally unknown. Authors also provide some results concerning the 
distribution of the limit random variable Z in some particular cases. In the present 
paper we continue the study of the model proving some central limit theorems and 
making another step toward the description of the distribution of Z. Further, the 
proven central limit theorems can be used in order to obtain asymptotic confidence 
intervals for the limit proportion Z. Moreover, we can also consider the case of 
more urns (for instance, according to the previous interpretations, the different urns 
can represent different markets or different populations), each of them following a 
HRRU dynamics, and perform some test for comparing them or get asymptotic 
confidence intervals for any linear combination of the limit proportions. 

The paper is organized as follows. In Section[^we formally introduce the model. 
In Section [3] we recall the needed facts concerning stable convergence and almost 
sure conditional convergence. In Section |4] we give and discuss the main results, 
whose proofs are postponed to Section [3 Finally, in Section El we provide some 
statistical tools based on the proven results. The paper is enriched with an appendix 
which contains some useful auxiliary results. 

2. The HRRU model 

An urn contains a S N \ {0} balls of color A and 6 S N \ {0} balls of color B. 
At each time n > 1, we simultaneously (i.e. without replacement) draw a random 
number Nn of balls. Let A„ be the number of extracted balls of color A. Then we 
return the extracted balls in the urn together with other i?„A„ balls of color A and 
Rn(Nn — Xn) balls of color B. More precisely, we take a probability space (U, A, P) 
and, on it, some random variables Nn, Xn, Rn such that, for each n > 1 , we have: 

i) The conditional distribution of the random variable Nn given 

[A^l, Ai, i?i, . . . , Nn-l, Xn-l, Rn-l] 
is concentrated on {I,..., S'„_i} where 

n—1 

Sn-i = a + b+ ^ NjRj = total number of balls at time n — I. (I) 

ii) The conditional distribution of the random variable Xn given 

[Ni, Xi, i?i, . . . , Nn-l,Xn-l, Rn-l, Nn] 
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is hypergeometric with parameters Nn, Sn-i and where 

n —1 

Hn-i = a + E XjRj = total number of balls of color A at time n — 1. (2) 

j=i 

iii) The random variable takes values in N \ {0} and it is independent of 
[A^l, Ai, i?i, . . . , Nn-l, A„_1, Rn-l, Nn, A„]. 

Note that we do not specify the conditional distribution of Nn given the past 
[-/Vi, Ai, i?i,..., A„_i, A„_i, Rn-i] nor the distribution of Rn- 

We will refer to the above urn model as the Hypergeometric Randomly Reinforced 
Urn (HRRu 2. It is worthwhile to remark that this model include the classical Polya 
urn (the case with A„ = 1 and = k for each n) and the randomly reinforced 
urn with the same reinforcements for both colors (the case with A„ = 1 for each n 
and Rn arbitrarily random). 

We set Zn equal to the proportion of balls of color A in the urn (immediately 
after the updating of the urn at time n and immediately before the (n + l)-th 
extraction), that is Zq = a/(a + b) and 

Zn = ^ for n > 1. 

Moreover we set 

Ao = {0,fl}, Rn=cr{Ni,Xi,Ri,...,Nn,Xn,Rn) for n > 1, 

and 

Gn = XnU a{Nn+l), Hn = Gn U a{R„+l) for 71 > 0. 

3. Stable convergence and almost sure conditional convergence 

Stable convergence has been introduced by Renyi in [42] and subsequently inve¬ 
stigated by various authors, e.g. [T||23l|28lIM]|39] . It is a strong form of convergence 
in distribution, in the sense that it is intermediate between the simple convergence 
in distribution and the convergence in probability. In this section we recall some 
basic definitions and properties. For more details, we refer the reader to |2S| [2S] 
and the references therein. 


Let (fl. A, P) be a probability space and let S' be a Polish space (i.e. a completely 
metrizable separable topological space), endowed with its Borel cr-field. A kernel 
on S, or a random probability measure on S, is a collection K = {A(w, •) : w S 17} 


^ We recall that a random variable X has hypergeometric distribution with parameter N, S, H 
if P{X = k}= 

It coincides with the model introduced in [3] but here the adopted notation is different: Mn 
in [3 corresponds to our Nn (total number of extracted balls at time n), Rn in [3] corresponds 
to our Xn (number of extracted balls of color A at time n) and Nn in [3] corresponds to our 
Rn (number of added balls for each extracted ball at time n). We decided to adopt a different 
notation with respect to [3] in order to use a notation more similar to the one used in the RRU 
model literature. 
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of probability measures on the Borel cr-field of S such that, for each bounded Borel 
real function / on S, the map 

u}^Kf{uj) = J f{x)K{uj,dx) 

is yl-measurable. Given a kernel on S and an event H in A with P{H) > 0, we can 
define a probability measure on S, denoted by PhK, as follows: 

PhK{B) = E[K{; B)\H] = P{Hy^ [ B) P{du:), 

Jh 

for each Borel set B oi S. We simply write PK when H = fl. It is easy to verify 
the relation 

J fix) PnKidx) = P{H)-^ J Kfiuj) P(dw). 

On (n, A, P) let (Yn) be a sequence of S'-valued random variables and let Khea. 

kernel on S. Then we say that converges stably to K, and we write T„ ^ 

if 

P(r„ e • I P) PhK for all P S ^ with P(iJ) > 0. 

Clearly, if Yn ^ then Yn converges in distribution to the probability mea¬ 

sure PK. Moreover, we recall that the convergence in probability of Yn to a random 
variable Y is equivalent to the stable convergence of Yn to a special kernel, which 
is the Dirac kernel K = Sy- 

We next mention a form of convergence, called almost sure conditional conver¬ 
gence, introduced and studied in m, and afterwards employed by other researchers 
(see, for example, pi H5]l. 

For each n, let P„ be a sub-cr-field of A and set P = (P„) (called conditioning 
system). If Kn denotes a version of the conditional distribution of given we 
say that Yn converges to K in the sense of the almost sure conditional convergence 
with respect to P, if, for almost every w in D, the probability measure P'„(w,-) 
converges weakly to K{LOy). Evidently, if Yn converges to K in the sense of the 
almost sure conditional convergence with respect to P, we have that 

E [fiYn) I P„] ^ Kf 

for each bounded continuous real function f on S and Yn converges in distribution 
to the probability measure PK. 

In the sequel we will adopt the notation A/'(0, V) in order to indicate the Gaussian 
kernel with zero mean and random variance V, that is the collection {A/'(0, V(uj)) : 
oj € D} of centered Gaussian distributions, where E is a positive random variable 
(A/'(0, 0) is meant as the Dirac probability measure concentrated in zero). Further, 
given two kernels Ki and K 2 , we will denote by Ki 0 K 2 the kernel given by the 
product measures Kiito, •) ® K 2 {uj, •). 
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4. Convergence results for the HRRU model 


The sequence {Zn) is a bounded 'H-martingale. Indeed, we have 


Zri , 1 - 


Rn{Xn — NnZn-l) 


( 3 ) 


and so 

E[Zr, - Z„_i\nn-l] = ^ {E[Xn\nn-l] - iVn^n-l) = ^ (£;[X„|a„-l] " 
= 0 


(where the second equality holds true because of condition iii) and the last one is 
implied by condition ii)). Hence, the sequence {Zn) converges almost surely (and in 
L^) to a random variable Z. Lemma I A. 2 1 (with = Xn/Nn) immediately implies 
that the sequence 


n ^ Nj 


(4) 


also converges almost surely (and in L^) to Z (cfr. Th. 3.1, Th. 3.5 in 


0 ). 


The distribution of Z is unknown except in a few particular cases (see 0)- We 
are going to prove the following central limit theorems, useful in order to get some 
information on Z. 

Theorem 1. Assume there exists a constant k G N \ {0} such that 7V„ V < k 
for each n and 

E[Nn\En-i] ^ N, E[Rn]^m, E[Rl]^q, (5) 

where N is a strictly positive hounded random variable and m and q are finite and 
strictly positive numbers. 

Then y/n{Zn — Z) converges in the sense of the almost sure conditional conver¬ 
gence with respect to E = (En) to the Gaussian kernel A/'(0, V), where 

V = qm-‘^N-^Z{l- Z). 

Theorem 2. Under the assumptions of Theorem\^ suppose also that 

E[N-^\En-i]^V. ( 6 ) 

where g is a strictly positive bounded random variable. 

Then 

[V^{Mn - Zn), V^{Zn - Z)] N{Q, U) ® N{0, H), 

where 

U = V + Z{1- Z){g- 2N-^) = +g- 2N-^) Z{1 - Z). 

From the above theorems we have that y/n{Mn — Zn) converges stably to A/’(0, U) 
and y/n{Mn — Z) converges stably to A/’(0, U -\-V). 


The following corollary enriches Corollary 3.4 in [3]. 

Corollary 3. Assume there exists a constant k G N \ {0} such that Nn V Rn < k 
for each n. Then: 

a) P{Z = Q) + P{Z = 1) < 1. 
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b) If assumptions are also satisfied, then P{Z = z) = 0 for all z G (0,1). 

Note that the above result entails that the limit Gaussian kernel in Theorem [1] 
is not degenerate. 

Some examples and comments follow. 

Example 4. If Nn = hn with G N \ {0} and hn f h < a + b, then the first 
condition in ([5]) and condition ([ 6 ]) are obviously satisfied with N = h and rj = h~^, 
so that we have V = Z(1 — Z) and U = — l)h~^Z{l — Z). 

Remark 5. If (fV„) is a sequence of integer-valued random variables with 1 < 
Nn < k and converging almost surely to a random variable N, then (by Lemma 
lA.ll) the first condition in ([5]) holds true. Moreover, condition ([ 6 ]) is satisfied with 
77 = N~^ and so we have U = {qm~'^ — 1)N~^Z(1 — Z). 

The next example concerns the above remark. 

Example 6. Suppose that (Nn) is given by a symmetric random walk with two 
absorbing barriers. More precisely, given h G N, with 2 < h < a + b, set 

n—l 

Ni=iG{2,...,h-l}, Nn=i + Y,^J 

where each Yj is independent of [Xi, Ri,Yi, X 2 , R 2 , ■ ■ ■ ,Yj-i, Xj, Rj] and such that 
P{Yj = -1) = P{Yj = 1) = 1/2. Set Ti = 0 and r„ = X;j=i Yj for n > 2, and 
define 

Ti = inf{n : Nn = 1 } = inf{n : r„ = 1 — i} 

Th = inf{n : iV„ = h} = inf{n : Tn = h — i}. 

Finally, for each n > 1, set Nn = NT/\n where T = Ti ATh- Then Nn N = Nt 
where N = I^t=Ti} + hI^T=Th}- fo order to find the probabilities P{T = Ti) = p 
and P{T = Th) = 1 — p, it is enough to observe that, since (r„) is a martingale, we 
have 

E[Tt] = (1 - i)p + (h - i){l - p) = 0 

and so p = {h — i)/{h — 1). According to Remark El p = N~^ = J{t=Ti} + 
h~^I{T=Th}- 

The last example regards the case when the random variables Nn are independent 
and identically distributed. 

Example 7. Suppose that {Nn) are a sequence of random variables such that each 
Nn is independent of Pn-i and uniformly distributed on the set { 1 ,..., /i}, with 
2 < h < a + b. Then N = E[Nn] = {h + l)/2 and p = E[N~^] = h~^ 

5. Proofs 

We begin with a preliminary result. 

Proposition 8. Assume there exists a constant fc G N \ {0} such that Nn V < k 
for each n and 


E[Nn\Pn-l] ^ N, E[Rn] 


m, 


(7) 
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where N is a strictly positive bounded random variable and m is a finite and strictly 
positive number. 

Then 


Sn 


Nm. 


n 


Proof. It follows from Lemma fA.2l with Yj = NjRj. Indeed, we have for 

each j and (by hi)) 


= E[NfiP,_,]E[Rfi ^ Nm. 


□ 


Proof of Theorem [H Setting Xf = X^/N^ for each n, the sequence (Xfi) is G- 
adapted and bounded. Moreover, we have 

EiXf+fiGr.] = E[Nfil,Xr.+, |a„] = N-l,E[X„+, |a„] = (8) 

and, as we have already said, the sequence (Z„) is a bounded C/-martingale. There¬ 
fore, in order to prove Theorem [Jl it sufhces to prove that the following conditions 
are satisfied (see Theorem lA.31 applied to Y^ = Xf): 

cl) L;[sup ^->1 v^l^j -1 - Zj\]< -foo; 

c2) ^ some random variable V. 

In the following we verify the above conditions. 


Condition cl). We observe that equality ([3]) can be rewritten as 

R,NfiZ,., - X') 


■^. 7-1 — 


s. 


so that we find 

\Z,_,-Zfi<j. 

Therefore condition cl) is obviously verified. 


(9) 

( 10 ) 


Condition c2). We want to apply Lemma I A. 2 1 with Yj = p{Zj-i — Zjfi. By 
the assumptions and inequality (US, we have "^E\Y^] < -foo. Moreover, by 

equality dH]), we have 

E[YfiP,.,] = fE[{Z,., - Zjf\E,_,] = fE[SfR]N]{Z,., - X'jf\P,.,l 
and so (by iii)) we get the two inequalities 

E[YfiP,_,] < ^E[R]]E[N]{Z,.,- X)f\P,_,]. 

Since Sn/n Nm and E[R^] converges to q, it is enough to prove the almost 
sure convergence of E[Nj{Zj-i — Xjfi\Pj-i] to NZ(I — Z). To this purpose, we 
observe that we can write 

E[Nf{Z,_, - = E [NfE[{Z,_, - \ T,.,] 











HYPERGEOMETRIC RRU 


9 


and, by ii) and relation ([8]), the conditional expectation E[{Zj-i — Xj)'^\Qj-i] coin¬ 
cides with 

+ N-^E[X]\g,_^] - 2Z,_^E[X'^\g,_^] = 

+ Nf - N,) + Z^^_^N]] - = 

Z,_i(l - Z,_i)(Vi - 1)-'^-' {S,-i - N,). 

Therefore we obtain 

E[Nf{Z,.,-X'f\E,-i] = {S,-^E[N,\E,-i]-E[Nf\E,-^]) , 

which converges to NZ{1 — Z) (since E[N'^\Tj-i] is bounded by and Sj-i 
-l-oo). Hence E\Yj\Tj-i\ converges almost surely to V and, by Lemma 1^31 condi¬ 
tion c2) is satisfied. 

The proof is so concluded. 

Proof of Theorem Thanks to what we have already proven in the previous 
proof, it suffices to verify that the following condition is satisfied (see Theorem lA.31 
applied to Yn = X^): 

c3) n~^ Sj=i ~ Zj-i +j{Zj-i — Zj)]"^ U for some random variable U. 

To this purpose, we apply Lemma I A. 2 1 with 

yj = [x'j-z,.^+j{z,_,-z,)]\ 

Indeed, by the assumptions and inequality (US, we have J2j j ^E[Y^] < -l-oo. 
Moreover, from what we have already seen in the previous proof, we can get 


E[{X'^-Z,_,)^\E,_,]E%r,Z{l-Z) 
and, with a similar arguments, 

2jE[{X'^ - Z,_,){Z,_, - Z,)\E,_,] = -2jE [Sj^ R,N,{Z,_^ - 

-2N-^Z{1 - Z). 

Proof of Corollary Assertion a) is proven in Corollary 3.4. in [3]. Let us prove 
assertion b) arguing as in [33] . 


Let A be a Vnevent and set /„ = E[lA\Xn]- Then /„ Ia- 
By Lemma lA.il we find 

E[ilA - /„)exp(^^yH(Z„ - Z))\Fn] ^ 0 . ( 11 ) 

On the other hand, by Theorem (TJ we have 

E[lneyrp{ity/n{Zn-Z))\En] = /„Ll[exp(it\/n(Z„-Z))|J'„] exp(-(t^l/)/2). 

( 12 ) 

Hence, from (ITTl) and (EH), we get 


E[Ia exp{ity/n{Zn - Z))\Eri] ^ exp(-(t^H)/2)/^. 


(13) 
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In order to conclude, it is enough to fix z G (0,1), take A = {Z = z} and observe 
that m implies almost surely 

/a exp(-(t^I/)/2) = \imE[lAexp{ity/n{Zn - Z))\En] 

n 

= limi?f/^exp(itv^(Z„ - z))\En] 

= lim/„ exp(zt-v/ri(Z„ — z)) = lim/^ exp^it^/ri^Zn — z)) 

n n 

and so almost surely 

Ia = |lim/Aexp(it-v/«(Z„ - z))| = exp(-(t^y)/2). 

n 

Since we have V > 0 on A, it results exp(—(t^F)/2) < 1 on A for t 0 and so we 
necessarily conclude that P{A) is zero. 


6. Statistical tools 


6.1. Asymptotic confidence intervals for the limit proportion. By means 
of Theorem [T] and Theorem [2J we can construct asymptotic confidence intervals for 
the limit proportion Z. More precisely, under the assumptions of Theorem [l] also 
assume k < a + b (so that < Sn-i for each n). If we are in the particular case 
when: 

• for each n, the random variable Nn is independent of J-n-i and all the 
random variables N„ are identically distributed with mean value /i (so that 
N = E[Nn] = p and -q = E[N~^]) and 

• all the random variables Rn (that are independent by assumption iii)) are 
also identically distributed (so that m = E[Rn] and q = 

then two asymptotic confidence intervals for Z are 


Zn ± gi-f 


Mn ± qi-% 


(14) 


where ( 7 i_“ is the quantile of order 1 — ^ of the standard normal distribution and 




^Zn{l - Zn), 


Wn = 




+qn-—j Mn(l - Mn) (15) 
minn k 


with 


mn = 

Qn — 

n 




Mn — 5 

Vn = 


n 

y" , N~ 

^.7 = 1 .7 


(16) 


n n 

Note that the second interval does not depend on the initial composition of the 
urn, which could be unknown. 


6.2. The case of more urns. Let U he a finite set. Every index u € U labels 
an urn initially containing a{u) balls of color A and b{u) balls of color B. Each of 
the urn follows the dynamics described in the Section [2j For instance, according to 
the interpretations given in Section [T1 we can see lA as a set of different markets or 
different populations. 
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More precisely, we take a probability space {n,A,P) and, on it, some random 
vectors Xn = Nn = [Nniu)]ueu, Rn = [Rn{u)\u(iU such that, for each 

n > 1, we have: 

i) The conditional distribution of the random vector iV„ given 

[Ni,Xi,Ri, . . . , Nn-l, Xn-l, Rn-l\ 
is concentrated on riuewil’ • ■ • ’ 'S'n-i(R)} where 

n—1 

Sn-i{u) = a{u) + b{u) + ^ Nj(u)Rj{u). ( 17 ) 

i=i 

ii) The conditional distribution of the random vector Xn given 

[iVi, Xi, i?i, . . . , Nn-l,Xn-l,Rn-l,Nn\ 
is the product 

Hypergeom(iV„(-«), 5'„_i('u), i7„_i('u)), 

uGU 

where Hypergeom(A^„(t6), S'„_i(t6), i7„_i(t6)) denotes the hypergeometric 
distribution with parameters N„(u), and Hn-i(u) with 

n — 1 

Hn-i{u) = a{u) + Y^X,{u)R,{u). ( 18 ) 

1=1 

hi) The random vector takes values in (N\{0})“’’'^*^^^ and it is independent 
of 

[A^l, Xl, i?i, . . . , iV„_i, Xn-l, Rn-1, Nn, Xn]- 
We set Zn(u) equal to the proportion of balls of color A in the urn u (immediately 
after the updating of the urn at time n and immediately before the (n + l)-th 
extraction), that is Zo{u) = a(u)/{a(u) + b{u)) and 

Zn{u) = for n > 1. 

Sn(u) 

Moreover we set 

7^0 = {0,11}, Fn = (y{Ni,Xi,Ri,...,Nn,Xn,Rn) fom > 1, 

and 

= 7„ V ct(7V„+i) forn > 0. 

From condition ii) follows that Xn(u) and A„(v) are t/„_i-conditionally inde¬ 
pendent for u ^ V and so, setting Xn(u) = Xn{u)/Nn(u) for each n and u, we 
have 

E [{Zn-l{u) - X'n{u)) {Zn-l{v) - X'n{v)) {Gn-l] = 

E[Zn-l{u) - X'^{u)\gn-l]E[Zn-l{v) - X'^lGn-l] ( 19 ) 

= 0 . 

It is worthwhile to note that, for a given n, we are not assuming the random 
variables Nn{u) (resp. i?„(w)), with u € U, to be independent. For example, we 
can assume 

Nniu) = h(u) + F'n Rn{u) = r{u) + E" 
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where h(u), r(u) are specific constants for each urn u and are random fac¬ 

tors that are common to all the urns. 


Suppose now that there exists k G N\ {0} with Nniu) V i?„ (m) < k < a{u) -I- b(u) 
for each n and u and that the additional assumptions stated in Section 16.11 are 
satisfied for each u. Set m{u) = q{u) = E[Rn{uY], ^{u) = E[Nn{u)] and 

r]{u) = E[Nn{u)~^]. Denoting by = [Mn{u)]ueu the vector containing the em¬ 
pirical mean of Xj{u) up to time n for each urn u and hy Z = [Z{u)]u^u the vector 
containing the almost sure limit of Zn(u) (and Mn{u)) for each u, we have as a con¬ 
sequence of (fT^ that, for any vector a = [a{u)]ueu of real numbers, the sequence 
y/n{a, {Zn - Z)fi converges in the sense of the almost sure conditional convergence 
withrespectto J'toA/'(0,X;„6w«(«)^^('“))>'^hereD('u) = 

and \/n{a, {Mn — Z)) converges stably to A/’(0, a(M)^(t/(w) -I- D(w))), where 

= { miSliu) + “ 7^) Similarly as done in the pre¬ 

vious section, these convergence results can be useful in order to get asymptotic 
confidence intervals for the linear combination (a, Z) of the limit proportions Z(u). 


Finally, the above results can be employed in order to obtain asymptotic critical 
regions for tests. For instance, in order to perform a statistical test with 

iJo : m{u) > card(W')“^ ^ m{v) against Eli : m{u) < card(W')“^ ^ m{v) 

v^W v^W 


where W dU and u^W, we can use the asymptotic critical region 
I y/card(^Y0-i 'Jn \Mr,{u) - Z„(m)| 


where 


with 


Un{u) = 


mn{u) = 
Uniu) = 


y/rUniu) 

quju) 

mn{u)^Hn{u) 




> qi-% 


Vniu) - 


n 


Mn(u) 
qniu) = 


Z„(u)(l-Z„(u)) (20) 


Y^URAu) 




rin{u) = 


( 21 ) 


Acknowledgments 

Irene Grimaldi acknowledges support from CNR PNR Project “CRISIS Lab”. More¬ 
over, she is a member of the Italian group “Gruppo Nazionale per I’Analisi Matem- 
atica, la Probabilita e le loro Applicazioni (GNAMPA)” of the Italian Institute 
“Istituto Nazionale di Alta Matematica (INdAM)”. 


References 

[1] AldouS, D. J. and Eagleson, G. K. (1978). On mixing and stability of limit theorems. Ann. 
Probab. 6, 325-331. 

[2] Aletti, G., May, G. and Secchi, P. (2009). A central limit theorem, and related results, 
for a two-color randomly reinforced urn. Adv. Appl. Probab. 41, 829—844. 


^ The symbol (■, ■) denotes the scalar product between two vectors. 















HYPERGEOMETRIC RRU 


13 


[3] Aoudia, D. a. and Perron, F. (2012). A new randomized Polya urn model. Applied Math¬ 
ematics 3, 2118-2122. 

[4] Bai, Z. D. and Hu, F. (2005). Asymptotics in randomized urn models. Ann. Appl. Probab. 
15, 914-940. 

[5] Bai, Z. D., Hu, F. and Rosenberger, W. F. (2002). Asymptotic properties of adaptive 
designs for clinical trials with delayed response. Ann. Statist. 30, 122—139. 

[6] Bassetti, F., Grimaldi, I. and Leisen F. (2010). Conditionally identically distributed 
species sampling sequences. Adv. Appl. Probab. 42, 433—459. 

[7] Beggs, a. W. (2005). On the convergence of reinforcement learning. J. Econom. Theory 
122(1), 1-36. 

[8] Berti, P., Grimaldi, I., Pratelli, L. and Rigo, P. (2015). Central limit theorems for an 
Indian buffet model with random weights. Ann. Appl. Probab. 25(2), 523—547. 

[9] Berti, P., Grimaldi, L, Pratelli, L. and Rigo, P. (2011). A central limit theorem and its 
applications to multicolor randomly reinforced urns. J. Appl. Probab. 48(2), 527—546. 

[10] Berti, P., Grimaldi, I., Pratelli, L. and Rigo, P. (2010). Central limit theorems for 
multicolor urns with dominated colors. Stoch. Proc. Appl. 120, 1473-1491. 

[11] Berti, P., Grimaldi, I., Pratelli, L. and Rigo, P. (2009). Rate of convergence of predictive 
distributions for dependent data. Bernoulli 15, 1351-1367. 

[12] Berti, P., Pratelli, L. and Rigo, P. (2004). Limit theorems for a class of identically 
distributed random variables. Ann. Probab. 32, 2029—2052. 

[13] Blackwell, D. and Dubins, L. (1962). Merging of opinions with increasing information. 
Ann. Math. Stat. 33(3), 882-886. 

[14] Boldi, P., Grimaldi, I. and Monti, C. (2014). A Network Model characterized by a Latent 
Attribute Structure with Competition. Available on arXiv (1407.7729, 2014), submitted. 

[15] Bose, A., Dasgupta, A. and Maulik, K. (2009). Multicolor urn models with reducible 
replacement matrices. Bernoulli, 15(1), 279-295. 

[16] Caldarelli, G., Chessa, A., Grimaldi, L, Pammolli, F. (2013). Weighted networks as 
randomly reinforced urn processes. Physical Review E 87(2), 020106(R). 

[17] Chauvin, B., Pouyanne, N. and Sahnoun, R. (2011). Limit distributions for large Polya 
urns. Ann. Appl. Probab. 21(1), 1-32. 

[18] Chen, M. R. and Kuba, M. (2013). On generalized Polya urn models. J. Appl. Probab. 
50(4), 1169-1186. 

[19] Chen, M. R. and Wei, C. Z. (2005). A New Urn Model. J. Appl. Probab. 42(4), 964-976. 

[20] COLLEVECCHIO, A., COTAR, C. AND LiCalzi, M. (2013). On a preferential attachment and 
generalized Polya’s urn model. Ann. Appl. Probab. 23(3), 1219-1253. 

[21] Grimaldi, I. (2009). An almost sure conditional convergence result and an application to a 
generalized Polya urn. Internat. Math. Forum 4(23), 1139-1156. 

[22] Grimaldi, I. and Leisen, F. (2008). Asymptotic results for a generalized Polya urn with 
multi-updating and applications to clinical trials. Communications in Statistics - Theory 
and Methods 37(17), 2777-2794. 

[23] Grimaldi, L, Letta, G. and Pratelli, L. (2007). A strong form of stable convergence. 
Seminaire de Probabilites XL (LNM 1899), Springer, 203-225. 

[24] Dasgupta, A. and Maulik, K. (2011). Strong laws for urn models with balanced replacement 
matrices. Electron. J. Probab. 16(63), 1723-1749. 

[25] Durham, S. D., Flournoy, N. and Li, W. (1998). A sequential design for maximizing the 
probability of a favourable response. Canad. J. Statist. 26, 479-495. 

[26] Eggenberger, F. and Polya, G. (1923). Uber die Statistik verketteter Vorgange. Zeitschrift 
Angew. Math. Mech. 3, 279-289. 

[27] Erev, I. AND Roth, A. (1998). Predicting how people play games: reinforcement learning in 
experimental games with unique, mixed strategy equilibria. Amer. Econ. Rev. 88, 848-881. 

[28] Feigin, P. D. (1985). Stable Convergence of Semimartingales. Stoch. Proc. Appl. 19, 125-134. 

[29] Hall, P. and Heyde, C. C. (1980). Martingale Limit Theory and Its Applications. Academic 
Press, New York. 

[30] Hopkins, E. and Posch, M. (2005). Attainability of boundary points under reinforcement 
learning. Games Econom. Behavior 53, 110-125. 

[31] Hu, F. AND Rosenberger, W. F. (2006). The Theory of Response-Adaptive Randomization 
in Clinical Trials. John Wiley and Sons Inc., New York. 



14 


IRENE GRIMALDI 


[32] Jacob, J. and Memin, J. (1981). Sur un type de convergence intermediaire entre la con¬ 
vergence en loi et la convergence en probabilite. Seminaire de Prohabilites XV (LNM 850), 
Springer, 529-546. 

[33] Janson, S. (2005). Limit theorems for triangular urn schemes. Probab. Theo. Rel. Fields 
134(3), 417-452. 

[34] Laruelle, S. and Pages, G. (2013). Randomized urn models revisited using stochastic 
approximation. Ann. Appl. Probab. 23(4), 1409-1436. 

[35] Mahmoud, H. (2008). Polya Um Models. Chapman-Hall, Boca Raton, Florida. 

[36] Martin, C. F. and Ho, Y. C. (2002). Value of information in the Polya urn process. Infor¬ 
mation Sciences 147, 65-90. 

[37] May, C. and Flournoy, N. (2009). Asymptotics in response-adaptive designs generated by 
a two-color, randomly reinforced urn. Ann. Statist. 37, 1058-1078. 

[38] Muliere, P., Paganoni, A. M. and Secchi, P. (2006). A randomly reinforced urns. J. 
Statisit. Plann. Inference 136(6), 1853—1874. 

[39] Peccati, G. and Taqqu, M. S. (2008). Stable convergence of multiple Wiener-Ito integrals. 
Journal of Theoretical Probability 21(3), 527—570. 

[40] Pemantle, R. (2007). A survey of random processes with reinforcement. Probab. Surveys 4, 


1-79. 


[41] Polya, G. (1931). Sur quelques points de la theorie des probabilite. Ann. Inst. Poincare 1, 
117-161. 

[42] Renyi, a. (1963). On stable sequences of events, Sankhya A 25, 293-302. 

[43] Zhang, L.X. (2014). A Gaussian process approximation for two-color randomly reinforced 
urns. Electron. J. Probab. 19(86), 1—19. 

[44] Zhang, L. X., Hu, F., Cheung, S. H. and Chan, W. S. (2014). Asymptotic properties of 
multicolor randomly reinforced Polya urns. Adv. Appl. Prob. 46, 585—602. 

[45] Zhang, L. X., Hu, F. and Cheung, S. H. (2006). Asymptotic theorems of sequential 
estimation-adjusted urn models for clinical trials. Ann. Appl. Probab. 16(1), 340-369. 


Appendix A. Some auxiliary results 


For reader’s convenience, we state here some results used in the proofs. 

Lemma A.l. (Th. 2 in [13] or a special case of Lemma A.2 in [2T| ') 

Let be a filtration and set J'oo = VnA- Then, for each sequence (F„) of 
integrable complex random variables, which is dominated in and which con¬ 
verges almost surely to a complex random variable Y, the conditional expectation 
E\Yn\Fn\ converges almost surely to the conditional expectation E\Y\Toc\- 

Lemma A. 2. (Lemma 2 in |5]) 

Let (Yn) be a sequence of real random variables, adapted to a filtration T. If 
^j>ij~'^E\Y^\ < - 1-00 and E\Yj\Tj-i\ Y for some random variable F, then 



Theorem A.3. (Special case of Th. 1 together with Prop. 1 in [^ and Th. 10 in 

m 

Let (Yn) be a bounded sequence of real random variables, adapted to a filtration 
G = {Gn)- Set 



i=i 

Suppose that (Z„) is a C/-martingale. 
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a.s. I Lr 


a.s.j L} 


Then, —>■ Z and Mn —>■ Z for some real random variable Z. Moreover, 

^/n{Zn — Z) converges in the sense of the almost sure conditional convergence with 
respect to G toward the Gaussian kernel Af{0, V) for some random variable V, 
provided 


cl) E [sup^->i \Z]-i - Zj\] < +00, 
c2) 

If condition 

c 3 ) E;=i - ^1-1 + jiZj-i - Z,)] " A U 
is also satisfied for some random variable U, then 


[v^ (M„ - Z„), v^(Z„ - Z)] Ar(o, u) G Ar(0, V). 
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